A Simple yet Effective Self-Debiasing Framework for Transformer Models

Current Transformer-based natural language understanding (NLU) models heavily rely on dataset biases, while failing to handle real-world out-of-distribution (OOD) instances. Many methods have been proposed to deal with this issue, but they ignore the fact that the features learned in different layers of Transformer-based NLU models are different. In this paper, we first conduct preliminary studies to obtain two conclusions: 1) both low- and high-layer sentence representations encode common biased features during training; 2) the low-layer sentence representations encode fewer unbiased features than the highlayer ones. Based on these conclusions, we propose a simple yet effective self-debiasing framework for Transformer-based NLU models. Concretely, we first stack a classifier on a selected low layer. Then, we introduce a residual connection that feeds the low-layer sentence representation to the top-layer classifier. In this way, the top-layer sentence representation will be trained to ignore the common biased features encoded by the low-layer sentence representation and focus on task-relevant unbiased features. During inference, we remove the residual connection and directly use the top-layer sentence representation to make predictions. Extensive experiments and indepth analyses on NLU tasks show that our framework performs better than several competitive baselines, achieving a new SOTA on all OOD test sets

Measuring the Quality of Service for High Occupancy Toll Lanes Operations

AbstractHigh Occupancy Toll (HOT) lane systems have been proposed as one of the most applicable countermeasures against freeway congestion. Under HOT lane operational scheme, a Single Occupancy Vehicle (SOV) can pay to access HOT lanes in exchange of travel time saving or enhanced trip reliability when excess HOT lane capacity is available. Compared with regular freeway facilities, HOT lane systems demonstrate unique characteristics in facility capacity, driver behavior, travel pattern, demand modeling, and trip reliability. This study aims at conducting a comprehensive performance analysis on two representative HOT lane systems of State Route 167 in Washington and I-394 MnPass in Minnesota based on the field data collected from traffic sensors and transponder toll tags. Performance measurements are proposed to quantify the quality of service for HOT lane operations. Three critical issues are addressed in this study: 1) the speed-flow relationships in HOT lane systems, 2) quantified system-wide travel time savings and travel time reliability achieved, 3) SOVs tolling incentives. Based on the empirical analysis and evaluation results for the SR 167 and I-394 MnPass HOT lane systems, operational problems and challenges are also identified. Although the HOT lane system preserves favorable travel reliability, under-utilized HOT lane capacities were observed. The existing tolling strategies may be modified for better SOV allocation for HOT lane usages and further optimize the overall HOT system operations. The research findings greatly advance our understanding on HOT lane system operation mechanisms and are complementary to the freeway facility performance analysis provided by Highway Capacity Manual 2000

Evidence for Dirac Fermions in a honeycomb lattice based on silicon

Silicene, a sheet of silicon atoms in a honeycomb lattice, was proposed to be a new Dirac-type electron system similar as graphene. We performed scanning tunneling microscopy and spectroscopy studies on the atomic and electronic properties of silicene on Ag(111). An unexpected $\sqrt{3}\times \sqrt{3}$ reconstruction was found, which is explained by an extra-buckling model. Pronounced quasi-particle interferences (QPI) patterns, originating from both the intervalley and intravalley scattering, were observed. From the QPI patterns we derived a linear energy-momentum dispersion and a large Fermi velocity, which prove the existence of Dirac Fermions in silicene.Comment: 6 pages, 4 figure